Apr 19, 2012
Robots.txt is a text (not html) file you place on your site to tell search robots which pages you would like them not to visit…